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Australian teachers, recruited via Facebook, completed an online survey about aspects of 
numeracy. The survey was designed to explore views on numeracy and capacity to respond 
to numeracy tasks. In this paper, we focus primarily on responses to two numeracy tasks - 
one numerical, the other requiring critical evaluation. On the first item, 40% answered 
correctly; on the second, 60% performed at a level expected of people aged 17 or older. The 
provocative findings warrant further research with a larger sample. 


Introduction 

In Australia it is expected that teacher education programs will prepare all graduates to 
“[Kjnow and understand literacy and numeracy teaching strategies and their application in 
teaching areas” (Australian Institute of Teaching and School Leadership [AITSL], 2014). 
There is a further expectation that all pre-service teachers will have achieved a specified 
level of numeracy (and literacy) proficiency prior to graduation; as from July 1 2016 it will 
be required that numeracy and literacy tests be passed in order to graduate. 

According to AITSL (2014), all proficient practicing teachers in Australia should be 
able to “[ajpply knowledge and understanding of effective teaching strategies to support 
students’ literacy and numeracy achievement”. To date, insufficient attention has been paid 
to the numeracy skill levels of those already in the profession. Data from a study in which 
this topic was explored are reported in this paper. 

Theoretical perspectives and previous research 

Geiger, Forgasz, and Goos (2015) defined numeracy as “the capacity to make effective 
use of mathematics in contexts related to personal life, the workplace, and in exercising civil 
responsibilities” (p. 611). This definition is consistent with the description of numeracy as 
one of seven general capabilities in the Australian Curriculum (Australian Curriculum, 
Assessment and Reporting Authority [ACARA], 2014). All teachers are charged with the 
responsibility of developing students’ numeracy capabilities across all disciplines areas 
within the Australian Curriculum and at all grade levels (ACARA, n.d.). 

The 21 st century numeracy model was initially designed as a framework to audit 
mathematics curriculum designs and was later used in a range of research projects for 
related purposes (Geiger et al., 2015). In particular, it has been adopted “to guide the design 
of numeracy tasks for implementation in subjects outside of mathematics” (Geiger et al., 
2015, p. 613). Context (real world setting) is central to the model. Mathematical knowledge, 
tools, and dispositions surround the context - they are considerations in the design of the 
numeracy task which is designed to develop personal and social, citizenship, and/or work- 
related numeracy skills. These dimensions of the model “are embedded within an 
overarching construct - critical orientation - an analytical and evaluative demand” (Geiger 
et al., 2015, p. 614). The numeracy model served as the framework for the development of 
the survey instrument used in this study, an earlier version of which had been administered 
to pre-service teacher education students at one university (see Leder, Forgasz, Kalkhoven, 
& Geiger, 2015). 


2016. In White, B., Chinnappan, M. & Trenholm, S. (Eds.). Opening up mathematics education research (Proceedings of the 
39th annual conference of the Mathematics Education Research Group of Australasia), pp. 238-245. Adelaide: MERGA. 
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Leder et al. (2015) were interested in pre-service teachers’ views about their proficiency 
in mathematics and the importance of mathematics for teaching, whether they recognised 
the mathematical demands in everyday life as well as on teachers in schools, apart from 
what students are taught. Over 50% of the respondents considered themselves to be good or 
excellent at mathematics, 76% agreed that it was important for teachers to be good at 
mathematics, and a large majority generally recognised the importance of mathematics and 
its applications in everyday life. Yet, only 44% believed that there were numeracy demands 
on teachers within the school that were outside the classroom. 

Callingham, Beswick, and Ferine (2015) reported on three separate secondary level 
contexts in which the development of teachers’ numeracy knowledge were central: a school 
focussing on project-based learning, teachers of subjects other than mathematics, and pre- 
service teachers formally studying a numeracy course. Their research findings highlighted 
two challenges: that all “teachers, including teachers of mathematics, need to understand 
and recognise the nature of numeracy” (p. 559), and that “teachers need to develop 
strategies to address numeracy in their subject areas in ways that are not about formal 
mathematics teaching” (p. 559). They felt that some progress had been made on the first 
challenge in the three case studies, but that the second proved more difficult. Systemic 
support over time was recommended. 

The Programme for the International Assessment of Adult Competencies (PIAAC) 
(OECD, n.d.a) was developed to assess adults’ (aged 16-65) literacy, numeracy, and 
problem-solving skills in technology-rich environments skills. Overall, Australians’ literacy 
skills were well above the OECD average, but their numeracy skills were slightly below 
(OECD, n.d.b). There are six levels of proficiency: below 1, 1-5. For numeracy, Australian 
data were: 6.5% below level 1, 15% at Level 1, 32% at Level 2, 31% at Level 3, 11% at 
Level 4, and 1.4% at Level 5 (ABS, 2014a). Among those with postgraduate degrees, 77% 
were at Level 3 or above; for those whose highest qualifications were Bachelor’s degrees, it 
was about 70% (ABS, 2014b). Teachers would fall into these two qualification categories; 
their anticipated numeracy skills should therefore generally be good. 

Biggs and Collis (1982) developed the SOLO (Structure of Observed Learning 
Outcomes) Taxonomy to provide information on students’ reasoning levels when engaged 
in mathematical problem-solving. Collis and Romberg (1992, p. 32) provide descriptions of 
the four main SOLO levels and how they apply in interpreting students’ responses to 
mathematical problem solving items, accompanied by the expected (average) age of 
performance at each level: 

Unistructural: use of one obvious piece of information from the problem stem (« 9 years of age) 

Multistructural: use of two or more separate pieces of information. . . (« 13 years of age) 

Relational: Use of an integrated understanding of two or more pieces of information... (« 17 years 

of age) 

Extended Abstract: Use of an abstract general principle or hypothesis which is derived from, or 

suggested by, the information. . . (» 17 + years) 

It should be noted that the SOLO taxonomy (Biggs & Collis, 1982) also described a pre- 
structural level, which was not included in Collis and Romberg (1992). In terms of 
problem-solving, a response at the pre-structural level would suggest that the respondent 
had misunderstood, or missed the point, of the question. 

In this study, we used the descriptors for the SOLO levels as a guide in detennining 
participants’ performance levels on one of the numeracy tasks, the Chips question (see 
below), that was included in our survey instrument. 
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The study 

In the study reported here, we sought to identify Australian teachers’ competence and 
confidence with numeracy tasks consistent with the 21 st century numeracy model. The two 
numeracy items (one numerical, the other involving critical evaluation) reported on in this 
paper are described in detail below. We were also interested to discern if there were 
differences among various sub-groups of teachers. Our specific research question and its 
three sub-questions were: 

How well do Australian teachers respond to numeracy items? 

a. For the numerical item, is there a difference in the accuracy rates of 
primary/secondary teachers, those who have/have not studied tertiary 
mathematics, or among older/younger teachers? 

b. How do confidence levels in the answers given compare for these same groups? 

c. For the critical evaluation item, what, and how much, information did the 
teachers focus on to justify their decisions? 

The instrument 

The instrument used in this study was a slightly modified version of the one piloted with 
pre-service teachers (see Leder et ah, 2015). The full instrument included biographical and 
teaching background items, and items probing views about mathematics and numeracy and 
their utility in a school setting and in everyday life. Also included were 
mathematical/numeracy items drawn from publicly available Australian grade 9 NAPLAN 
(National Assessment Program for Literacy and Numeracy) tests 1 , items (with permission) 
from the pool of released PISA 2 items, and two items developed by the researchers and 
colleagues. When appropriate, respondents were asked to elaborate on the answer they had 
given. Qualtrics (www.qualtrics.com) was used to prepare the instrument for online 
completion. To comply with space constraints, only those items of interest to the study 
reported here are included in paper. 

Data gathering 

To obtain a broad sample of Australian teachers (as well as teachers from elsewhere in 
the world), an advertisement to recruit participants was placed on Facebook; Forgasz, Leder, 
and Tan (2014) provide detailed infonnation on the method for doing so. Since the AITSL 
numeracy (and literacy) standard applies to all teachers, primary or secondary, including 
those who do not teach mathematics, all Australian teachers were invited to complete the 
online survey. 

The sample 

Of the 1198 who responded to the Facebook advertisement and clicked on the link to the 
survey, 1040 went further and completed the items in which biographical data were 
requested: sex; age; school (country) location, type and level in which they were working; 
years of teaching experience; whether they had studied mathematics at the tertiary level; and 


1 NAPLAN is the National Assessment Program for Literacy and Numeracy which, since 2008, has been 
administered to all Australian students in grades 3, 5, 7 and 9. 

2 PISA is administered every three years to samples of 15 year old students in many countries around the 
world, including Australia. 
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whether mathematics was among the subjects they taught. Approximately half of the 1040 
respondents persisted and provided responses to all or most of the remaining items. 

In this paper we focus on those who nominated Australia as their place of work 
(N=100). The respondents comprised 84 females and 16 males. Over two-thirds (N=71) 
were aged over 40. Most (N=72) taught in a government school, 21 in a non-government 
school, and seven in a different (e.g., tertiary or early childhood) setting. A high percentage 
(80%) indicated that mathematics was among the subjects they taught. The majority of these 
were teaching in a primary school where, as is common in Australian schools, mathematics 
is among the subjects taught. 

Over 60% (N=64) of the group persisted and worked through the whole survey. Because 
of occasional response omissions the sample size for examination of the research questions 
listed above varied between 58 and 64 (see Table 1 in the Results section). The two groups, 
those who finished the survey and those who did not, generally resembled each other on the 
descriptors listed above, with the inclusion of primary teachers being a noticeable exception; 
more than three quarters of those who taught at the secondary level, 26 out of 3 1, completed 
the survey, compared with approximately half of those working in a primary setting, 34 out 
of 63. 

Analyses 

Descriptive statistics were used to describe the sample and the teachers’ performance 
levels on the first numeracy item. To compare the responses of the various sub-groups of 
teachers, cross-tabulations and chi-square tests were conducted. For the second numeracy 
item, frequency counts were accompanied by categorisations of responses, based on the 
numbers of pieces of information used to justify the choice of chips brand, into the SOLO 
taxonomy levels of performance. 


Results 

In this paper, we focus on those who responded to two numeracy questions included in 
the online survey. The first was taken from the 2010 grade 9 NAP LAN test and required a 
numerical response to be supplied. The other, devised by the researchers and colleagues, 
required a decision to be made and justified by drawing on the information provided. 

Numeracy question 1 - the “Keypad question ” 

The wording of the grade 9 NAPLAN item (the “keypad question”) was as follows: 

Helen’s office has security alarm. To turn it off Helen has to type her 4-digit security code into this 
keypad. [A diagram of a 10-digit keypad was included.] Helen’s code is 0051. Including Helen’s 
code, how many different 4-digit codes are possible? 

To ascertain respondents’ confidence in the answers they provided, we included the 
following question: 

Do you think your answer is correct: [Yes / No / Unsure] 

For this item, we explored response patterns by age, teaching level (primary/secondary), 
and whether mathematics had been studied at the tertiary level (yes/no). Since the majority 
of respondents were female, and because so many were primary teachers who all teach 
mathematics, analyses were not conducted by gender or by whether or not they taught 
mathematics. 
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Overall performance for this item 

Sixty participants provided answers to the keypad question, 24 (40%) provided the 
correct answer (10,000), and 36 (60%) were incorrect with an amazing range of answers. 

The sample sizes for the different groups being compared on the keypad question are 
shown in Table 1. 

Table 1 

Respondents for keypad question by category 


Category 

Correct 

Incorrect 

Thought 

correct 

Thought 

incorrect 

Unsure 

Age < 40 

7 

11 

9 

2 

8 

>40 

17 

25 

24 

4 

17 

Teach Primary 

9 

22 

13 

3 

18 

Secondary 

14 

12 

19 

3 

4 

Tertiary mathematics studied 

14 

10 

18 

1 

7 

not studied 

10 

26 

15 

5 

18 


Participants ’ confidence in their answers - overall and by respondent age 

Although only 60 provided answers to the keypad question, 64 participants indicated 
whether they did (or would) get the question correct. Of the 64, 33 (52%) thought that their 
answer was correct, 6 (9%) expected it to be incorrect, and 25 (39%) were unsure. There 
was no statistically significant difference in the responses by participants’ age (those aged 
40 or under; those aged over 40). 

Participants ’ answers and confidence in their answer - by whether or not 
mathematics had been studied at university 

A higher proportion of those who had studied mathematics at the tertiary level answered 
the question correctly: 14 (58%) compared with 10 (29%) of those who had not. The 
corresponding data for those who were incorrect were 10 (42%) and 26 (72%) respectively. 
A test of the frequency distribution of these responses revealed a statistically significant 
difference: yfi = 5.6, p = .02, (p = .31 

Having studied mathematics at university also seemed to have an effect on the 
confidence of the answer given: 18 (70%) of those who indicated that they had studied 
mathematics at that level thought their answer to have been correct compared with 15 (40%) 
of those who had not. Corresponding figures for those who were uncertain were 7 (27%) 
and 18 (47%) respectively. Few of those who answered thought that their answer was 
incorrect: 1 (4%) and 5 (13%) respectively. A chi-square test of the frequency distribution 
of this set of responses produced a value just short of statistical significance: % 2 2 = 5.73, p = 
.057, (p = .30. More of those who studied mathematics at university were confident that they 
could answer this item correctly. 

Participants ’ confidence in their answer - by level taught ( primary or secondary ) 

As can be seen from Table 1, the sample comprised 34 primary school teachers and 26 
secondary teachers. For these groups, those who thought they had answered the question 
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correctly, incorrectly, or were unsure were, respectively: 13 (38%) and 19 (73%); 3 (9%) 
and 3 (12%); and 18 (53%) and 4 (15%). A chi-square test of the frequency distribution of 
these responses revealed a statistically significant difference: % 2 2 = 9.13, p = .01, (p = .39. 
Those who taught at the secondary level were more confident that their answer was correct; 
a much higher proportion of those teaching at the primary level were unsure. 

The relatively small sample precludes more detailed analyses of the results. 

Numeracy question 2 - the “chips question ” 

The second numeracy question of interest paper was the “chips question”. Participants 
were provided with the images of three different packets of vinegar-flavoured chips - Red 
Rock, Smiths, and Kettle. The dietary information, as found on each packet, was 
summarised in tabular form. Participants were told that “Some people are concerned about 
the nutritional value and costs of snack food such as chips”. The questions asked of 
participants were: 

You want to by a packet of salt and vinegar chips. Which brand of chips (of those described) would 

you buy? Explain why you made this choice and include some mathematics in your justification. 

Clearly there is no correct response to which brand to buy. Of interest were the 
justifications provided for the choices made. 

There were 64 responses with explanations for the brand of chips chosen. The brands 
selected were as follows: Red Rock: 24; Smiths: 2; and Kettle (38). The number of bits of 
infonnation (nutritional, cost, other) that were provided to justify the brand chosen varied 
from one to 7. The distribution of brands selected by number of pieces of information 
included in justifications is shown in Table 2. 


Table 2 

Brand of chips selected by number of pieces of information included in justification 


Number of pieces of infonnation 

Brand 

1 

2 

3 

4 

5 

6 

7 

Missed point 

Red Rock 

1 

2 

7 

4 

4 

3 

3 

- 

Smiths 

- 

1 

- 

- 

- 

- 

- 

1 

Kettle 

2 

17 

11 

2 

4 

1 

- 

1 

TOTAL 

3 

20 

18 

6 

8 

4 

3 

2 


The frequencies of the types of information (nutritional, cost, other) mentioned by 
respondents in their justifications of brand choice are summarised in Table 3. 


Table 3 

Frequencies of information type included in the justifications for brand choice 


Sugar 

Salt 

Fat 

Fibre 

Price 

Energy 

Protein 

22 

27 

36 

11 

50 

11 

5 

Net weight 

Cholesterol 

Carbohydrates 

Potassium 

Personal 

circumstance 

Like 

Depends 

25 

3 

6 

1 

3 

2 

1 
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As can be seen in Table 3, the most frequently mentioned factors were: price (mentioned 
in 50 of the justifications), Fat (36), Salt/Sodium (27), net weight (25), and sugar (22). It 
should be noted that when cost was mentioned it was either as “the cheapest” option or that 
cost was not as important a factor as nutritional value. 

Using the SOLO taxonomy levels as a guide, the justifications could be categorised 
within each SOLO taxonomy level. We used the following for SOLO categorisations: 

Prestructural: misses the point Unistructural: uses 1 piece of information 

Multistructural: uses 2 pieces of information Relational: uses 3/4 pieces of information 

Extended abstract: uses 5+ pieces of information 

The frequencies of responses at each SOLO level (see Table 2) and representative 
examples of the responses provided are shown in Table 4. 


Table 4 

Samples of responses representing each SOLO level 


SOLO level N 

Prestructural 2 

Unistructural 3 

Multistructural 20 

Relational 24 


Extended 1 5 

abstract 


Sample responses 

There is a 1 in 3 chance of choosing the correct chips, only one 
pack is labelled Salt and vinegar 

Lower sugar [Red Rock] 

The Kettle ones, as they are cheapest and have approx 2 kilojoules 
per gram less than the Red Rock Deli. [Kettle] 

Better value for money. Cheaper, and more weight. [Kettle] 

I chose Red rock deli because it has the lowest sugar per lOOg. It 
also has the most fibre and the lowest salt content per 100 gram 
serve as well. [Red Rock] 

Cheapest, Lowest Kilojoules, Less Fat than the second cheapest, 
High Sodium content, but I happen to know they taste good too. 
Red Rock appear to be the healthier option, but they are also the 
most expensive. [Kettle] 

I made my choice based on the dietary infonnation, not price. The 
dietary information for Red Rock Deli more suited my diet as it has 
the lowest rates of fat, cholesterol, sugars and sodium. It also has 
the highest level of dietary fibre. If I was looking purely at price 
and packet size, and not dietary infonnation, I would have 
purchased the Kettle brand. [Red Rock] 


Based on the data in Table 2, the mean number of pieces of infonnation per response 
was 3.2 (204 pieces of infonnation for 64 people); arguably, the group of teachers were at 
the relational SOLO level, expected of 17 year olds. Using the average age at which each 
SOLO level is anticipated, together with the frequencies of responses at the SOLO levels 
shown in Table 4, the teachers’ overall perfonnance could also be determined. These data 
suggest that only those at the Relational and Extended abstract levels (39 in total, or 60%) 
are perfonning as expected of those aged 17 and above; and 25 (40%) are below that level. 
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Final words 


The data base for the work reported in this paper comprised a volunteer sample of 
practising teachers who responded to a survey on the numeracy capabilities of teachers, 
advertised on Facebook. Practical considerations led to a focus on the subset teaching in 
Australian schools and on their responses to two numeracy items. The first was an item 
included in a NAPLAN test administered to students in Year 9 in 2010. For our sample, 
those who had studied mathematics at university had a higher expectation of being able to 
answer the question correctly, and indeed a higher proportion answered the question 
correctly. Age, with 40 taken as the arbitrary point of division, did not significantly affect 
the expectation of being able to solve the item correctly. Those who taught at the secondary 
level, but not necessarily mathematics, were more confident that their answers were correct 
than those teaching at the primary level, even though for most of the latter group, 
mathematics was included among the subjects being taught. Only 40% answered this item 
correctly. Data from the second item yielded more nuanced information about the practising 
teachers’ performance levels. Using the SOLO taxonomy to assess performance, 60% were 
found to be performing at a level expected of those aged at least 17. On the basis of these 
items, and for our small sample of Australian teachers, numeracy proficiencies appear to 
warrant closer scrutiny. The size and self-selection of our sample precludes making credible 
generalizations about the broader teaching force. The findings are sufficiently provocative, 
however, to serve as a solid justification for more intensive study. 
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